Monitoring system, monitoring device, and monitoring method

ABSTRACT

A monitoring system, a monitoring device, and a monitoring method are provided. The monitoring system includes a detection part that detects a position of a worker intruded into a work area, a first specifying part that specifies a worker movable area on the basis of a position of the worker, an image capturing part that captures an image of an area including at least the worker movable area and a predetermined robot occupied area, a second specifying part that specifies a robot movable area from the image of the area, a third specifying part that specifies a human body area of the worker from the image of the area, a measuring part that measures the distance between the robot movable area and the human body area, and a restricting part that restricts movement of the robot when the distance is equal to or less than a predetermined distance.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Japanese PatentApplication no. 2017-025602, filed on Feb. 15, 2017. The entirety of theabove-mentioned patent application is hereby incorporated by referenceherein and made a part of this specification.

BACKGROUND OF THE INVENTION Field of the Invention

The disclosure relates to a monitoring system, a monitoring device, anda monitoring method for monitoring the movement of workers and robotssuch that the workers can work safely within a work area where workersand robots work.

Description of Related Art

A monitoring system that monitors the movement of workers and robotswithin a work area where workers and robots work in collaboration hasbeen proposed in the related art. For example, in a technology describedin Patent Document 1, a plurality of image capturing devices captureimages of the entire work area and image processing is performed on thecaptured images to detect the positions of workers and the positions ofrobots. The technology described in Patent Document 1 controls themovement of robots such that workers can work safely on the basis of thedetected positions of the workers and the detected positions of therobots.

In the above Patent Document 1, there is a need to perform imageprocessing on images of the entire work area, and therefore there is aproblem that the burden of image processing increases.

RELATED ART DOCUMENT Patent Documents

[Patent Document 1] Published Japanese Translation No. 2006-501487 ofthe PCT International Publication

SUMMARY OF THE INVENTION

In one or some of exemplary embodiments of the invention, a monitoringsystem, a monitoring device, and a monitoring method that reduce theburden of image processing are provided.

In one or some of exemplary embodiments of the invention, a monitoringsystem having a monitoring device configured to monitor movement of aworker and a robot to allow the worker to work safely in a work area inwhich the worker and the robot work includes a detection part configuredto detect a position of the worker that has intruded into the work area,a first specifying part configured to specify a worker movable areaindicating a range in which the worker is movable on the basis of theposition of the worker detected by the detection part, an imagecapturing part configured to capture an image of an area including atleast the worker movable area specified by the first specifying part anda predetermined robot occupied area of the robot, a second specifyingpart configured to specify a robot movable area indicating a range inwhich a movable portion of the robot is movable according to the imageof the area captured by the image capturing part, a third specifyingpart configured to specify a human body area of the worker according tothe image of the area captured by the image capturing part, a measuringpart configured to measure a distance between the robot movable areaspecified by the second specifying part and the human body areaspecified by the third specifying part, and a restricting partconfigured to restrict movement of the robot when the distance measuredby the measuring part is equal to or less than a predetermined distance.

In one or some of exemplary embodiments of the invention, a monitoringdevice configured to monitor movement of a worker and a robot to allowthe worker to work safely in a work area in which the worker and therobot work includes a first specifying part configured to specify aworker movable area indicating a range in which the worker is movable onthe basis of a position of the worker detected by a detection partconfigured to detect the position of the worker that has intruded intothe work area, a second specifying part configured to specify a robotmovable area indicating a range in which a movable portion of the robotis movable according to an image of an area captured by an imagecapturing part configured to capture the image of the area including atleast the worker movable area specified by the first specifying part anda predetermined robot occupied area of the robot, a third specifyingpart configured to specify a human body area of the worker according tothe image of the area captured by the image capturing part, a measuringpart configured to measure a distance between the robot movable areaspecified by the second specifying part and the human body areaspecified by the third specifying part, and a restricting partconfigured to restrict movement of the robot when the distance measuredby the measuring part is equal to or less than a predetermined value.

In one or some of exemplary embodiments of the invention, a monitoringmethod for monitoring movement of a worker and a robot to allow theworker to work safely in a work area in which the worker and the robotwork includes specifying a worker movable area indicating a range inwhich the worker is movable on the basis of a position of the workerdetected by a detection part configured to detect the position of theworker that has intruded into the work area, specifying a robot movablearea indicating a range in which a movable portion of the robot ismovable according to an image of an area captured by an image capturingpart configured to capture the image of the area including at least thespecified worker movable area and a predetermined robot occupied area ofthe robot, specifying a human body area of the worker according to theimage of the area captured by the image capturing part, measuring adistance between the robot movable area and the human body area, andrestricting movement of the robot when the measured distance is equal toor less than a predetermined value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an exemplary configuration of a monitoringsystem 1 according to an embodiment.

FIG. 2 is a diagram showing a hardware configuration of a controller.

FIG. 3 is a diagram showing an exemplary functional configuration of amonitoring device.

FIG. 4 is a diagram showing a worker movable area.

FIG. 5 is a diagram showing a robot occupied area.

FIG. 6 is a diagram showing a region where a worker movable area and arobot occupied area overlap.

FIG. 7 is a diagram showing a combined area 600B.

FIG. 8 is a diagram showing a robot movable area.

FIG. 9 is a diagram showing a robot movable area and a human body area.

FIG. 10 is a diagram showing an aggregate of cluster information.

FIG. 11 is a diagram showing areas including a work area.

FIG. 12 is a diagram showing a flowchart for a monitoring device.

FIG. 13 is a diagram showing a method of detecting a movement error.

FIG. 14 is a diagram showing a method of calculating coordinateconversion parameters.

FIG. 15 is a diagram showing an exemplary functional configuration of amonitoring device according to a modification.

FIG. 16 is a diagram showing another method for specifying the positionof a worker.

FIG. 17 is a diagram showing another method for specifying the positionof an arm.

DESCRIPTION OF THE EMBODIMENTS

In one or some of exemplary embodiments of the invention, a monitoringsystem, a monitoring device, and a monitoring method that reduce theburden of image processing are provided.

In one or some of exemplary embodiments of the invention, a monitoringsystem having a monitoring device configured to monitor movement of aworker and a robot to allow the worker to work safely in a work area inwhich the worker and the robot work includes a detection part configuredto detect a position of the worker that has intruded into the work area,a first specifying part configured to specify a worker movable areaindicating a range in which the worker is movable on the basis of theposition of the worker detected by the detection part, an imagecapturing part configured to capture an image of an area including atleast the worker movable area specified by the first specifying part anda predetermined robot occupied area of the robot, a second specifyingpart configured to specify a robot movable area indicating a range inwhich a movable portion of the robot is movable according to the imageof the area captured by the image capturing part, a third specifyingpart configured to specify a human body area of the worker according tothe image of the area captured by the image capturing part, a measuringpart configured to measure a distance between the robot movable areaspecified by the second specifying part and the human body areaspecified by the third specifying part, and a restricting partconfigured to restrict movement of the robot when the distance measuredby the measuring part is equal to or less than a predetermined distance.

In one or some of exemplary embodiments of the invention, the firstspecifying part is configured to specify a new worker movable area byexcluding an area in which the worker cannot be present from the workermovable area specified on the basis of the position of the workerdetected by the detection part.

In one or some of exemplary embodiments of the invention, the monitoringsystem further includes a setting part configured to set an area that isa combination of the worker movable area and the robot occupied areathat at least partially overlaps the worker movable area as a combinedarea, wherein the second specifying part is configured to specify therobot movable area according to an image of the combined area in thearea captured by the image capturing part, and the third specifying partis configured to specify the human body area according to the image ofthe combined area in the area captured by the image capturing part.

In one or some of exemplary embodiments of the invention, the imagecapturing part is configured to capture an image of only a combined areathat is a combination of the worker movable area and the robot occupiedarea that at least partially overlaps the worker movable area.

In one or some of exemplary embodiments of the invention, the secondspecifying part is configured to specify the robot movable area on thebasis of the marker provided on the movable portion of the robot.

In one or some of exemplary embodiments of the invention, the monitoringsystem further includes a fourth specifying part configured to specify aposition of the movable portion of the robot according to an imagecaptured by the image capturing part on the basis of a marker providedon the movable portion of the robot, and a comparing part configured tocompare the position of the movable portion of the robot specified bythe fourth specifying part with a movement indicated by movementinformation input to a controller of the robot.

In one or some of exemplary embodiments of the invention, the robot isshiftable, the detection part is configured to specify a position of therobot, and the first specifying part is configured to specify the robotoccupied area on the basis of the position of the robot detected by thedetection part.

In one or some of exemplary embodiments of the invention, a monitoringdevice configured to monitor movement of a worker and a robot to allowthe worker to work safely in a work area in which the worker and therobot work includes a first specifying part configured to specify aworker movable area indicating a range in which the worker is movable onthe basis of a position of the worker detected by a detection partconfigured to detect the position of the worker that has intruded intothe work area, a second specifying part configured to specify a robotmovable area indicating a range in which a movable portion of the robotis movable according to an image of an area captured by an imagecapturing part configured to capture the image of the area including atleast the worker movable area specified by the first specifying part anda predetermined robot occupied area of the robot, a third specifyingpart configured to specify a human body area of the worker according tothe image of the area captured by the image capturing part, a measuringpart configured to measure a distance between the robot movable areaspecified by the second specifying part and the human body areaspecified by the third specifying part, and a restricting partconfigured to restrict movement of the robot when the distance measuredby the measuring part is equal to or less than a predetermined value.

In one or some of exemplary embodiments of the invention, a monitoringmethod for monitoring movement of a worker and a robot to allow theworker to work safely in a work area in which the worker and the robotwork includes specifying a worker movable area indicating a range inwhich the worker is movable on the basis of a position of the workerdetected by a detection part configured to detect the position of theworker that has intruded into the work area, specifying a robot movablearea indicating a range in which a movable portion of the robot ismovable according to an image of an area captured by an image capturingpart configured to capture the image of the area including at least thespecified worker movable area and a predetermined robot occupied area ofthe robot, specifying a human body area of the worker according to theimage of the area captured by the image capturing part, measuring adistance between the robot movable area and the human body area, andrestricting movement of the robot when the measured distance is equal toor less than a predetermined value.

According to the monitoring system, the monitoring device, and themonitoring method according to the present technology, it is possible toreduce the burden of image processing.

Embodiments of the invention will be described in detail with referenceto the drawings. In the drawings, the same or corresponding parts aredenoted by the same reference numerals and descriptions thereof will notbe repeated.

<Exemplary Configuration of Monitoring System 1>

First, an exemplary configuration of a monitoring system 1 including amonitoring device according to the present embodiment will be described.

FIG. 1 is a diagram showing an exemplary configuration of the monitoringsystem 1 according to the present embodiment. As shown in FIG. 1, themonitoring system 1 according to the present embodiment has a monitoringdevice 100, one or more cameras 4 serving as one or more image capturingparts, and one or more sensors 2 serving as one or more detection parts.The monitoring device 100 includes a vision controller 120 and a safetycontroller 150. In the present embodiment, the vision controller 120,the safety controller 150, and a robot controller which will bedescribed later are provided independent of each other. In other words,the vision controller 120, the safety controller 150, and the robotcontroller are configured in separate housings. In the example of FIG.1, two cameras 4 are installed and two sensors 2 are installed.

The monitoring system 1 monitors the movements of workers 300 and robots400 such that the workers 300 can work safely within a work area α inwhich the workers 300 and the robots 400 work. In the presentembodiment, the robots 400 operate (i.e. work) in a state in which theyare fixed at their locations without shifting. The sensors 2 detect thepositions of the workers 300 who have intruded into the work area α. Inthe present embodiment, radio frequency (RF) tags that output radiowaves are used as a method of detecting the positions of the workers300. The sensors 2 can detect the RF tags. The workers 300 attach RFtags 302 to parts of their bodies, whereby the sensors 2 specify thepositions of the RF tags 302 (i.e., the positions of the workers 300).The positions of the workers 300 are, for example, three-dimensionalcoordinates in the work area α. The “sensors 2 specify the positions ofthe workers 300” means, for example, that “the sensors 2 acquirethree-dimensional coordinates of the RF tags 302 attached to the workers300 in the work area α.”

Tags that output ultrasonic waves may be used instead of the RF tags. Inthis case, the sensors 2 detect the positions of the RF tags 302 byreceiving ultrasonic waves. What the tags output and the sensor 2receive are not limited to radio waves and ultrasonic waves and may besomething other than radio waves and ultrasonic waves.

A plurality of robots 400 are installed in the work area α. In theexample of FIG. 1, the robots 400 are placed on a mounting table 402.Within the work area α, the workers 300 can work in collaboration withthe robots 400.

A structure 500 is installed in the work area α. The structure 500 maybe one which the workers 300 cannot enter. The structure 500 may be, forexample, a control device including a robot controller that controls therobots 400. The structure 500 may be a shelf on which loads of theworkers 300 or the like can be placed.

The cameras 4 mainly capture moving images of the workers 300, therobots 400, or the like in the work area α. Each of the cameras 4includes an optical system such as a lens and an image capturingelement. The image capturing element is a coupled charged device (CCD),a complementary metal oxide semiconductor (CMOS) sensor, or the like. Inthe present embodiment, two cameras 4 are provided. Through the twocameras 4, it is possible to three-dimensionally capture images in thework area α.

<Exemplary Hardware Configuration of Monitoring Device 100>

FIG. 2 shows an example of the exemplary hardware configuration of thevision controller 120. As shown in FIG. 2, the vision controller 120includes a central processing unit (CPU) 101 that executes programs, aread only memory (ROM) 102 that stores data in a nonvolatile manner, arandom access memory (RAM) 103 that stores data in a volatile manner, aflash memory 104, an input part 107, a communication interface (IF) 108that transmits and receives information and signals, and a notifyingdevice 109 that performs error notification or the like. A programexecuted by the CPU 101 or the like is stored in the ROM 102. The safetycontroller 150 and the robot controller respectively adopt the samehardware configuration as that of FIG. 2.

<Exemplary Functional Configuration of Monitoring Device 100>

Next, an exemplary functional configuration of the monitoring device 100is shown. FIG. 3 is a diagram showing the exemplary functionalconfiguration of the monitoring device 100. The vision controller 120includes a second specifying part 8, a third specifying part 10, afourth specifying part 16, and a setting part 17. The safety controller150 includes a first specifying part 6, a measuring part 12, arestricting part 14, a comparing part 18, and a light emission controlpart 154.

The sensors 2 detect the position of a worker 300 who has intruded intothe work area α on the basis of an RF tag 302 attached to the worker300. Detection information indicating the position of the worker 300detected by the sensors 2 is input to the first specifying part 6. Thefirst specifying part 6 specifies a worker movable area 300A on thebasis of the detection information indicating the RF tag 302 (theposition of a worker 300) detected by the sensors 2.

FIG. 4 is a diagram showing the worker movable area 300A. The workermovable area 300A indicates a range in which the worker 300 is movable.In general, the movement speed of a worker within the work area α ispresumed. The worker movable area 300A is an area that is estimated tobe that through which the worker 300 or a part thereof (for example, anam of the worker 300) passes as the worker 300 or the part thereof moveswithin a predetermined time (for example, 1 second).

The worker movable area 300A is an area including the position of the RFtag 302 detected by the sensors 2. Therefore, even when a detectionerror of the sensors 2 has occurred, the worker movable area 300A canabsorb the detection error. Thus, the worker movable area 300A of thepresent embodiment is an area that also takes into considerationdetection errors of the sensors 2. The worker movable area 300A can alsobe said to be an area in which there is a possibility that there is anintruder (that is, a worker 300). A region of interest (ROI) in theimages captured by the cameras 4 is an area of interest to be monitoredin the image captured by the cameras 4 and corresponds to the workermovable area 300A specified by the first specifying part 6.

The worker movable area 300A specified by the first specifying part 6 isinput to the setting part 17. In this specification, a term “area” suchas in “worker movable area 300A” is also used to refer to informationindicating the area. Therefore, for example, the “worker movable area300A is input to the setting part 17” means that “information indicatingthe worker movable area 300A is input to the setting part 17.” A robotoccupied area 400A stored in the RAM 103 is also input to the settingpart 17.

Here, the robot occupied area 400A will be described. FIG. 5 is adiagram showing the robot occupied area 400A. The robot occupied area400A is an area through which a part of a robot 400 (for example, arms406 shown in FIG. 8) and the entirety of the robot 400 may pass duringall work that the robot 400 may execute. The robot occupied area 400A isalso information stored in a predetermined storage area (in the RAM 103in the present embodiment). The robot occupied area 400A is apredetermined area since the position of each robot is predetermined.The robot occupied area 400A is an area specified by a range ofthree-dimensional coordinates (an x coordinate, a y coordinate, and a zcoordinate).

A manager of the monitoring device 100, for example, a supervisor of thework area α or the like, inputs the position of each robot 400 to theinput part 107. A conversion part (not specifically shown) included inthe monitoring device 100 converts the input position of each robot 400into a robot occupied area. The converted robot occupied area is storedin the RAM 103 for each robot.

Generally, the position of each robot is changed depending on which workis performed in the work area α. The robot occupied area 400A isgenerated from the position of each robot 400 input by the manager ofthe monitoring device 100. Thus, even if the position of the robot ischanged, the monitoring device 100 can generate the robot occupied area400A flexibly according to the change.

The setting part 17 sets an area that is a combination of the workermovable area 300A provided from the first specifying part 6 and therobot occupied area 400A that at least partially overlaps the workermovable area 300A as a combined area. FIG. 6 is a diagram showing aworker movable area 300A and a robot occupied area 400A which at leastpartially overlaps the worker movable area 300A. In the example of FIG.6, the worker movable area 300A and a part of the robot occupied area400A overlap each other in an area 600A (the hatched portion). When oneof the worker movable area 300A and the robot occupied area 400A isincluded in the other, the greater of the worker movable area 300A andthe robot occupied area 400A may be set as a combined area.

FIG. 7 is a diagram showing a combined area 600B set by the setting part17. In the example of FIG. 7, the setting part 17 sets an area obtainedby excluding an exclusion area 500A from an area corresponding to thelogical sum (i.e., the OR) of the worker movable area 300A and the robotoccupied area 400A as the combined area 600B. This exclusion area 500Ais the area of the structure 500. The worker 300 cannot enter thestructure 500. Thus, the exclusion area 500A can be said to be an areawhere the worker 300 cannot be present. The set combined area 600B isinput to the second specifying part 8, the third specifying part 10, andthe fourth specifying part 16. In the present embodiment, the settingpart 17 excludes the exclusion area 500A from the worker movable area300A and sets an area corresponding to the logical sum of the workermovable area 300A, excluding the exclusion area 500A, and the robotoccupied area 400A as the combined area 600B.

The position of the structure 500 is also predetermined. Thus, theexclusion area 500A of the structure 500 is learned in advance andstored in advance in a predetermined storage area (for example, in theRAM 103). The setting part 17 excludes the exclusion area 500A from theworker movable area 300A using information (coordinates) of the storedexclusion area 500A. Here, any method may be used for the learning orthe like. Such a method is described, for example, in the specificationof “United States Patent Application Publication No. 2011-242283.”

The second specifying part 8 specifies a robot movable area indicating arange, in which a movable portion of a robot 400 (an a in thereof in thepresent embodiment) is movable, according to the image of an areacaptured by the cameras 4. The robot 400 is that which is included inthe combined area 600B in the image of the area captured by the cameras4. FIG. 8 is a diagram showing the robot movable area.

First, a detailed structure of the robot 400 will be described withreference to FIG. 8. The robot 400 includes a base portion 404 and arms406. The arms 406 include a first arm 411, a second arm 412, a third arm413, a joint portion 414 connecting the first arm 411 and the second arm412, a joint portion 415 connecting the second arm 412 and the third arm413, and a joint portion 416 connecting the third arm 413 and the baseportion 404. This configuration allows the arms 406 to move in threedimensions.

The second specifying part 8 specifies a robot movable area on the basisof markers provided on the arms 406 which are movable portions of therobot 400. The markers of this embodiment are LEDs that emit light. Inthe present embodiment, a first LED 421 is provided at a center portionin the extending direction of the first arm 411. A second LED 422 isprovided at a center portion in the extending direction of the secondarm 412. A third LED 423 is provided at a center portion in theextending direction of the third arm 413. Each of the LEDs emits lightunder the control of the light emission control part 154 in the safetycontroller 150.

Each LED may be attached to each arm in a wound form such that the LEDdoes not enter positions where the LED cannot be captured by the cameras4 as the LED moves into the shadow of the robot 400 due to movement ofthe robot 400.

The first LED 421, the second LED 422, and the third LED 423 emit lightin respective light emission modes associated therewith. Table 1 is adiagram showing the associated light emission modes. In the example ofTable 1, the first LED 421 continuously emits light (i.e. continues toemit light), the second LED 422 emits light every one second (i.e.blinks every one second), and the third LED 423 emits light every twoseconds (i.e. blinks every two seconds). The light emission modes of thefirst LED 421, the second LED 422, and the third LED 423 (hereinafterreferred to as “LED light emission modes”) are stored in a predeterminedstorage area (the ROM 102 in the present embodiment).

TABLE 1 LIGHT EMISSION LED MODES. FIRST LED 421. CONTINUOUSLY EMIT.SECOND LED 422. BLINK EVERY ONE SECOND. THIRD LED 423. BLINK EVERY TWOSECONDS.

The image captured by the cameras 4 is input to the second specifyingpart 8. The second specifying part 8 extracts an image included in thecombined area 600B from the captured image. Thereafter, the secondspecifying part 8 can recognize movements of the first arm 411, thesecond arm 412, and the third arm 413 in the extracted image on thebasis of the LED light emission modes stored in the ROM 102. That is,the second specifying part 8 can recognize a movement of the first arm411 by recognizing an LED which continuously emits light as the firstLED 421 and then recognizing the trajectory of the first LED 421. Thesecond specifying part 8 can recognize a movement of the second arm 412by recognizing an LED which blinks every one second as the second LED422 and then recognizing the trajectory of the second LED 422. Thesecond specifying part 8 can recognize a movement of the third arm 413by recognizing an LED which blinks every two seconds as the third LED423 and then recognizing the trajectory of the third LED 423.

By recognizing the movement of the first arm 411, the second specifyingpart 8 can recognize a first arm area 411A in which the first arm 411 ismovable (i.e., an area presumed to be that in which the first arm 411 ispresent). By recognizing the movement of the second arm 412, the secondspecifying part 8 can recognize a second arm area 412A in which thesecond arm 412 is movable (i.e., an area presumed to be that in whichthe second arm 412 is present). By recognizing the movement of the thirdarm 413, the second specifying part 8 can recognize a third arm area413A in which the third arm 413 is movable (i.e., an area presumed to bethat in which the third arm 413 is present).

Further, the second specifying part 8 specifies a robot movable area400B by combining the first arm area 411A, the second arm area 412A, andthe third arm area 413A. In the present embodiment, the secondspecifying part 8 specifies an area corresponding to the logical sum(OR) of the first arm area 411A, the second arm area 412A, and the thirdarm area 413A as the robot movable area 400B. The robot movable area400B is a region indicating a range in which the arms 406 are movable onthe basis of the movement of the movable portions (i.e., the arms 406)of the robot 400. In other words, the robot movable area 400B is an areaestimated to be that through which the arms 406 of the robot 400 pass asthey move. The robot movable area 400B is also an area including thepositions of the LEDs recognized by the second specifying part 8.Therefore, even when a recognition error of the second specifying part 8has occurred, the robot movable area 400B can absorb the recognitionerror. The robot occupied area 400A is an area through which a part ofthe robot 400 (for example, the arms 406 shown in FIG. 8) and theentirety of the robot 400 may pass during all work that the robot 400may execute. On the other hand, the robot movable area 400B is an areathrough which it is predicted that the arms 406 will pass on the basisof the current movements of the arms 406. Therefore, the robot movablearea 400B is narrower than the robot occupied area 400A.

The second specifying part 8 can specify the robot movable area 400Bindicating a range in which the movable portions of the robot 400 moveby recognizing movements of the LEDs (i.e., the first LED 421, thesecond LED 422, and the third LED 423) included in the combined area600B in the captured image. The specified robot movable area 400B isinput to the measuring part 12. The robot movable area 400B is an areaspecified by a range of three-dimensional coordinates (an x coordinate,a y coordinate, and a z coordinate).

The third specifying part 10 specifies a human body area of the worker300 according to an image of the combined area 600B in the area of theimage captured by the cameras 4. FIG. 9 is a diagram showing a robotmovable area 400B and a human body area 300B in an image inside thecombined area 600B. For example, the third specifying part 10 extractsfeature amounts on a minimum image unit basis (for example, on a pixelbasis) of the image of the combined area 600B. Feature amounts of thehuman body are stored in advance as a model in the ROM 102. Here, thefeature amounts are those which indicate the face of the worker 300,work clothes worn by the worker 300, and the like. The third specifyingpart 10 calculates the degrees of matching between the extracted featureamounts and the feature amounts stored in the ROM 102 according to thedifferences therebetween. For example, the degree of matching calculatedby the third specifying part 10 is small if the difference is great, andgreat if the difference is small. The third specifying part 10specifies, as a human body area, a region corresponding to featureamounts whose matching degrees are higher than a predetermined thresholdvalue. It should be noted that the method of specifying the human bodyarea is not limited to this and other methods may be used.

The third specifying part 10 may also extract a feature amount of anarea of the RF tag 302 and a feature amount of an area around the RF tag302 in the combined area 600B without extracting feature amounts ofother areas. Compared to the configuration in which feature amounts ofall regions of the combined area 600B are extracted, this configurationcan narrow the area from which feature amounts are extracted, therebyreducing the burden of processing for extracting feature amounts. Thespecified human body area 300B is input to the measuring part 12. Thehuman body area 300B is an area specified by a range ofthree-dimensional coordinates (an x coordinate, a y coordinate, and a zcoordinate).

The measuring part 12 measures the distance L between the robot movablearea 400B specified by the second specifying part 8 and the human bodyarea 300B specified by the third specifying part 10. For thismeasurement, the measuring part 12 divides the human body area 300B intoregions of minimum units to generate an aggregate 300C of clusterinformation. FIG. 10 is a diagram showing the aggregate 300C of clusterinformation. The measuring part 12 measures the (three-dimensional)distance L between the robot movable area 400B and a cluster informationitem 300Ca which is closest to the robot movable area 400B in theaggregate 300C of cluster information. The measured distance L is inputto the restricting part 14.

When the input distance L is equal to or less than a predetermineddistance Lc (threshold), the restricting part 14 transmits a restrictingsignal to the robot controller 200 to restrict the movement of the robot400. The robot controller 200 is a controller that controls the movementof each robot 400. The restricting signal is a signal for restrictingthe movement of the robot 400.

Here, the robot 400 whose movement is to be restricted (hereinafter alsoreferred to as a “robot to be controlled”) is a robot whose distance Lto the cluster information item 300Ca is determined to be equal to orless than the distance Lc. Examples of the restriction of the movementof the robot include a process of lowering the movement speed of therobot 400 and a process of completely stopping the movement of the robot400. The predetermined distance Lc is stored in advance in the ROM 102.The distance Lc is, for example, 1 m.

For example, upon determining that the distance L is equal to or lessthan the distance Lc and further that the difference ΔL between thedistance L and the distance Lc is equal to or less than a firstpredetermined value Th1, the restricting part 14 transmits a restrictingsignal, which indicates that the movement speed of the robot 400 is tobe lowered, to the robot controller 200. Thus, upon receiving therestricting signal, the robot controller 200 can lower the movementspeed of the target robot 400, the movement of which is to berestricted.

Further, upon determining that the distance L is equal to or less thanthe distance Lc and further that the difference ΔL between the distanceL and the distance Lc is equal to or less than a second predeterminedvalue Th2 (Th2<Th1), the restricting part 14 transmits a restrictingsignal, which indicates that the movement of the robot 400 is to becompletely stopped, to the robot controller 200. Thus, upon receivingthe restricting signal, the robot controller 200 can completely stop themovement of the target robot 400 whose movement is to be restricted.

As described above, upon determining that the distance L is equal to orless than the distance Lc, the monitoring device 100 controls the robot400 differently according to the distance L between the worker 300 andthe robot 400. More specifically, for example, the monitoring device 100performs control to lower the movement speed of the robot 400 if thedistance L between the worker 300 and the robot 400 is relatively great.Also, the monitoring device 100 performs control to completely stop themovement of the robot 400 if the distance L between the worker 300 andthe robot 400 is short.

[Advantageous Effects of Monitoring System 1]

Next, advantageous effects of the monitoring system 1 of the presentembodiment will be described. In brief, the monitoring system 1specifies a rough position of a worker 300 and performs image processingonly around the specified position. More specifically, the monitoringsystem 1 specifies a robot movable area 400B (see FIG. 8) and a humanbody area (see FIG. 9) by performing image processing according to animage of a combined area 600B in an image captured by the cameras 4.Thereafter, when the distance L between the robot movable area 400B andthe human body area 300B is equal to or less than the predetermineddistance Lc, the movement of the robot 400 to be controlled isrestricted in the monitoring system 1. In this manner, an image forwhich whether or not the movement of the robot 400 is to be restrictedis determined (hereinafter referred to as an “image on which imageprocessing is to be performed”) can be limited (narrowed) to the imageof the combined area. Therefore, according to the monitoring system 1 ofthe present embodiment, it is possible to reduce the burden of imageprocessing, compared to a known monitoring system in which suchdetermination is made for the entire image of a work area.

In addition, the third specifying part 10 specifies the human body area300B by processing only the ROI in the image captured by the cameras 4and therefore it is possible to lower the processing load for specifyingthe human body area 300B. Since the processing load can be reduced, themonitoring system 1 can also increase the frame rate of an object forimage processing. Therefore, it is possible to improve the responseperformance of the monitoring system 1.

Further, the monitoring system 1 according to the present embodimentdetects only a robot included in the combined area 600B, that is, arobot spaced away from the worker 300 by a distance equal to or lessthan the distance Lc to restrict the movement of the robot. In otherwords, the monitoring system 1 does not restrict movement of robotsother than the detected robot. Therefore, it is possible to improve theworking ratio of robots within the work area α, compared to a monitoringsystem that restricts the movement of all robots within the work area α.

Furthermore, the combined area 600B which is an area to be subjected toimage processing is obtained by excluding the exclusion area 500A froman area corresponding to the logical sum of the worker movable area 300Aand the robot occupied area 400A. Therefore, compared to “the monitoringsystem in which image processing is performed on an area correspondingto the logical sum of the worker movable area 300A and the robotoccupied area 400A from which the exclusion area 500A is not excluded”,it is possible to narrow the area of an image to be subjected to imageprocessing, thereby reducing the burden of image processing.

Moreover, the monitoring system 1 of the present embodiment measures thedistance L between the robot movable area 400B narrower than the robotoccupied area 400A and the human body area 300B (i.e., the aggregate300C of cluster information). Therefore, it is possible to narrow thearea of an image which is to be subjected to image processing whenmeasuring the distance L, compared to “the monitoring system thatmeasures the distance L between the robot occupied area 400A wider thanthe robot movable area 400B and the human body area 300B”. Thus, it ispossible to reduce the burden of image processing.

In addition, the monitoring system 1 of the present embodiment canspecify the robot movable area 400B on the basis of markers (the firstLED 421, the second LED 422, and the third LED 423 in the presentembodiment) attached to the arms 406 of the robot 400. Therefore, themonitoring system 1 of the present embodiment can accurately specify therobot movable area 400B.

[Areas]

Next, provided areas other than the work area α will be described. FIG.11 is a diagram showing areas including the work area α. In the exampleof FIG. 11, each area is viewed from directly above. In the example ofFIG. 11, an intrusion detection area β is provided in an area includingthe entire work area α. Further, a warning area γ is provided in an areaincluding the entire intrusion detection area β.

The warning area γ is an area where warning notification is performed bythe notifying device 109 upon intrusion of an intruder irrespective ofwhether or not an RF tag 302 is attached to the intruder. The intrusionmay be detected by the cameras 4 and may also be detected by a detectiondevice (not shown) other than the cameras 4. The intrusion detectionarea β is an area where the sensors 2 can detect/authenticate anintruder or the like having an RF tag 302 attached thereto. Apredetermined warning is output when a worker who has intruded into theintrusion detection area β fails to be detected/authenticated due to nothaving an RF tag 302 or the like. In this case, the monitoring system 1may stop the work system (for example, stop controlling the robot 400)in view of safety.

[Flowchart]

Next, a flowchart for the monitoring device 100 will be described. FIG.12 is a diagram illustrating a flowchart for the monitoring device 100.Each step is performed by each component included in the monitoringdevice 100.

In step S2, the first specifying part 6 performs a detection process todetect intrusion of workers by starting detection in the intrusiondetection area β through the sensors 2. In step S4, if it is determinedthat intrusion of a worker into the intrusion detection area β has beendetected through the detection process of step S2 (YES in step S4), theprocess proceeds to step S6.

In step S6, the first specifying part 6 detects detection targets of thesensors 2. In step S8, the first specifying part 6 determines whether ornot detection targets of the sensors 2 have been detected through thedetection process of step S6. If the determination result is NO in stepS8, the process proceeds to step S32. In step S32, the restricting part14 restricts the movement of all robots 400 by transmitting a controlsignal to the robot controller 200. This restriction involves completelystopping the movement of all robots 400. When it is determined in stepS8 that detection targets of the sensors 2 have not been detectedalthough it is determined in step S4 that intrusion of a worker into theintrusion detection area has been detected, it is highly likely that atleast one of the sensors, the monitoring device 100, and the like isfaulty. Thus, in such a case, safety can be guaranteed by completelystopping the movement of all robots 400.

In step S10, the first specifying part 6 detects the position of aworker 300 through the sensors 2. Next, in step S12, the firstspecifying part 6 specifies (sets) a worker movable area 300A on thebasis of coordinate conversion parameters which will be described later.Next, in step S14, the setting part 17 performs a process of narrowingthe worker movable area 300A using three-dimensional structureinformation of the structure 500 according to the worker movable area300A. The three-dimensional structure information is informationincluding three-dimensional coordinates of the structure 500. Thesetting part 17 excludes the exclusion area 500A from the worker movablearea 300A as the narrowing process.

Next, in step S16, the setting part 17 sets a combined area 600B using arobot occupied area 400A. Next, in step S18, the second specifying part8 detects light emission of LEDs (a first LED 421, a second LED 422, anda third LED 423) provided in arms 406 of a robot 400 in the combinedarea 600B. Next, in step S20, if the second specifying part 8 determinesthat light emission of LEDs is detected through the detection of stepS18 (YES in step S20), the second specifying part 8 determines that therobot with the detected LEDs attached thereto is a “robot to becontrolled” in step S22. On the other hand, if the determination resultis NO in step S20, the process returns to step S10.

In step S24, the second specifying part 8 specifies a robot movable area400B on the basis of the positions of the detected LEDs. Next, in stepS26, the third specifying part 10 specifies a human body area 300B (anaggregate 300C of cluster information) in the combined area 600B. Next,in step S28, the measuring part 12 calculates a (three-dimensional)distance L between the human body area 300B (the aggregate 300C ofcluster information) and the robot movable area 400B. Next, in step S30,the restricting part 14 determines whether or not the calculateddistance L is equal to or less than a predetermined distance Lc. If thedetermination result is YES in step S30, the movement of the robot to becontrolled is restricted in step S32. On the other hand, if thedetermination result is NO in step S30, the process returns to step S10.

[Movement Error of Robot]

Next, a method of detecting a movement error of a robot will bedescribed. FIG. 13 is a diagram illustrating the method of detecting amovement error. The method of detecting a movement error will bedescribed with reference to FIGS. 3 and 14.

As shown in FIG. 13, the fourth specifying part 16 of the visioncontroller 120 includes a light emission pattern specifying part 124 anda robot position specifying part 122. The robot controller 200 includesa control clock generating part 202 and a robot control commandgenerating part 204.

The control clock generating part 202 generates a control clock. Thegenerated control clock is input to the robot control command generatingpart 204 and the light emission control part 154. The robot controlcommand generating part 204 generates a command signal on the basis ofthe clock signal and transmits the command signal to the robot 400. Therobot 400 moves on the basis of the command signal.

The light emission control part 154 transmits a light emission controlsignal to the robot 400 on the basis of the clock signal. Here, thelight emission control signal is a signal indicating the light emissionmode of Table 1. The first LED 421, the second LED 422, and the thirdLED 423 emit light on the basis of the light emission control signal. Inthis manner, the first LED 421, the second LED 422, and the third LED423 are caused to emit light and the robot 400 is caused to move on thebasis of the clock signal and therefore “light emission of the first LED421, the second LED 422, and the third LED 423” and “movement of therobot 400 (the arms 406)” can be synchronized with each other.

Further, the light emission control part 154 transmits light emissionpattern information shown in Table 1 to the comparing part 18. The robotcontrol command generating part 204 transmits robot position informationto the comparing part 18. The robot position information is positioninformation of each arm (i.e., information indicating the position ofeach arm in the work area α) that the robot control command generatingpart 204 provides in a command to the robot 400.

The robot position specifying part 122 specifies (measures) thepositions of the arms 406 on the basis of light emission of the firstLED 421, the second LED 422, and the third LED 423 according to an imagecaptured by the cameras 4. The robot position specifying part 122transmits the robot position specification information to the comparingpart 18. The robot position specification information is informationindicating respective actual positions (respective behaviors) of theLEDs provided in each arm (information indicating each LED specified bythe robot position specifying part 122).

The light emission pattern specifying part 124 specifies the lightemission pattern of each LED according to the image captured by thecameras 4. Here, although the light emission pattern specifying part 124can specify the light emission pattern, it cannot specify which of theLEDs corresponds to the specified light emission pattern. The lightemission pattern specifying part 124 and the robot position specifyingpart 122 associates the light emission pattern specification informationwith a behavior of an LED that emits light in the light emission patternspecified from the light emission pattern specification information(i.e., the robot position specification information), and transmit themto the comparing part 18. “Movement information” output from the fourthspecifying part in FIG. 3 includes robot position specificationinformation and light emission pattern specification information.

The comparing part 18 compares the light emission pattern informationfrom the light emission control part 154 with the light emission patternspecification information from the light emission pattern specifyingpart 124. This allows the comparing part 18 to recognize an LEDcorresponding to the actual light emission pattern (the light emissionpattern specified by the light emission pattern specifying part 124).For example, when the actual light emission pattern is continuous lightemission, the comparing part 18 recognizes that the LED continuouslyemitting light is the first LED 421. Similarly, the comparing part 18recognizes the other LEDs, i.e., the second LED 422 and the third LED423. Thereafter, the comparing part 18 compares the robot positionspecification information and the robot position information for eachrecognized LED and determines whether or not a behavior (positioninformation) of the recognized LED matches. As an example of thisdetermination process, the comparing part 18 calculates the differencebetween a numerical value of the robot position specificationinformation and a numerical value of the robot position information. Thecomparing part 18 determines that there is no difference in the positioninformation with each other if the difference is less than a thresholdvalue and determines that there is a difference in the positioninformation with each other if the difference is equal to or greaterthan the threshold value.

Upon determining that there is a difference, the comparing part 18determines that there is an abnormality in control of the robot 400 orthe like and transmits an abnormality signal indicating that there is anabnormality to the notifying device 109. The notifying device 109performs abnormality notification on the basis of the abnormalitysignal. For example, when the comparing part 18 determines that theposition of an arm of the robot is abnormal, the notifying device 109displays “there is an abnormality in the arm.” The movement of the robotdetermined to be abnormal by the comparing part 18 may be restricted.According to this configuration, abnormality notification is performedwhen it is determined that at least one of the arms of the robot 400 isabnormal. Therefore, it is possible to guarantee the worker's safety.

Further, the monitoring system 1 may restrict a robot for which thepresence or absence of a movement error is to be detected to thatincluded in the combined area 600B. According to this configuration, itis possible to reduce the processing for detecting the presence orabsence of a movement error, compared to the configuration in whichpresence or absence of a movement error is detected for all robots inthe work area α. In the monitoring system 1, presence or absence of amovement error may be detected for all robots in the work area α.According to this configuration, it is possible to guarantee the safetyof workers 300 present in the work area α.

Note that a movement error can be similarly detected for a shiftablerobot although only the case in which robots 400 are fixed has beendescribed above.

[Coordinate Conversion Parameters]

Next, an exemplary method of calculating coordinate conversionparameters described in step S12 of FIG. 12 will be described. Bycalibrating a coordinate system of the sensors 2 and a coordinate systemof the vision controller 120, it is possible to calculate coordinateconversion parameters. The coordinate conversion parameters areparameters that the first specifying part 6 uses to specify the workermovable area 300A according to the position of the worker 300 detectedby the sensor 2.

FIG. 14 is a diagram illustrating the method of calculating coordinateconversion parameters. First, a reference marker 601 for the origin andreference markers 602, 603, and 604 for other points are arranged in thework area α. Further, detection targets of the sensor 2 are arranged tooverlap the reference markers 601 to 604. In the present embodiment, thedetection targets are RF tags. RF tags 3021, 3022, 3023, and 3024 arearranged to overlap the reference markers 601 to 604, respectively. Thenumber of reference markers for the other points is not limited to threeand may be other numbers, for example, more than three.

The vision controller 120 extracts the reference markers 602 to 604 forthe other points to construct a first reference plane H1. The firstreference plane H1 is a plane including the reference markers 602 to604. Further, the vision controller 120 extracts the reference marker601 for the origin to specify a first coordinate origin P1 in the firstreference plane H1.

Next, the safety controller 150 constructs a second reference plane H2on the basis of detection information (three-dimensional positioninformation) obtained from the sensors 2 that have detected the RF tags3022 to 3024 arranged to overlap the reference markers 602, 603, and 604for the other points, respectively. Further, the safety controller 150specifies a second coordinate origin P2 on the basis of the detectioninformation (three-dimensional position information) obtained from thesensors 2 that have detected the RF tag 3021 arranged to overlap thereference marker 601 for the origin. Thereafter, a monitoring part (notspecifically shown) included in the monitoring device 100 calculatescoordinate conversion parameters on the basis of the first referenceplane H1, the first coordinate origin P1, the second reference plane H2,and the second coordinate origin P2. The calculated coordinateconversion parameters are stored in a predetermined storage area (forexample, the RAM 103).

[Modification]

Although the above embodiment of the invention has been described withreference to the drawings, the invention is not limited to the aboveembodiment. The invention is applicable to various modifications andapplications without being limited to the above embodiment. Hereinafter,modifications or the like applicable to the invention will be described.

[Captured Image]

In the above embodiment, the cameras 4 capture all images within thework area α. In this modification, the cameras 4 capture only a partcorresponding to the combined area 600B. FIG. 15 is a diagram showing anexemplary functional configuration of a monitoring device 110 of thismodification. When FIG. 3 and FIG. 15 are compared, the difference isthe configuration of the vision controller. A vision controller 121 ofthis modification (FIG. 15) is different from that of FIG. 3 in that thesetting part 17 in the vision controller 120 shown in FIG. 3 is replacedby a camera instruction part 123.

The camera instruction part 123 obtains a combined area 600B from aworker movable area 300A, a robot occupied area 400A, and the like. Thecamera instruction part 123 transmits an instruction signal, whichallows the cameras 4 to capture images of only the combined area 600B,to the cameras 4. On the basis of the instruction signal, the cameras 4capture images of only the combined area 600B. The instruction signal isthat which indicates the positions of the cameras 4, the directions ofthe optical axes of the lenses of the cameras, the focuses of thecameras 4, and the like. The captured images, that is, the imagescaptured only for the combined area 600B, are input to the secondspecifying part 8, the third specifying part 10, and the fourthspecifying part 16. The second specifying part 8 specifies a robotmovable area of a robot 400 included in the combined area 600B. Thethird specifying part 10 specifies a human body area of a worker 300included in the combined area 600B. The fourth specifying part 16specifies movement information of the robot 400 included in the combinedarea 600B. The subsequent processes are similar to those of the aboveembodiment and thus the descriptions thereof will not be repeated.

According to the configuration of this modification, images of only thecombined area 600B are captured, such that it is possible to reduce thenumber of cameras compared to the monitoring system that captures imagesof all regions of the work area α. Further, according to theconfiguration of this modification, the cameras 4 capture images when acombined area is generated, such that it is possible to reduce theburden of imaging processing compared to the monitoring system in whichthe cameras 4 always capture images.

[When Robot is Shiftable]

The invention can also be applied to the case in which at least one ofthe robots in the work area α is shiftable. This will be described belowwith reference to FIG. 3.

In this case, the sensors 2 detect the position of the shiftable robot.As a method of detecting the position of the shiftable robot, forexample, an RF tag may be attached to the shiftable robot and thesensors 2 may detect the RF tag. The frequency of radio waves from theRF tag attached to the worker 300 may be set different from thefrequency of radio waves from the RF tag attached to the shiftablerobot. This allows the monitoring system to clearly specify the worker300 and the shiftable robot without confusion.

Thereafter, the first specifying part 6 specifies both a worker movablearea and a robot occupied area of the shiftable robot. The workermovable area and the robot occupied area specified by the firstspecifying part 6 are input to the setting part 17. The setting part 17specifies a combined area of the worker movable area and the robotoccupied area. The subsequent processes are similar to those of theabove embodiment and thus the descriptions thereof will not be repeated.

According to this configuration, it is possible to guarantee safety ofthe worker 300 even when a shiftable robot is present in the work areaα.

[RF Tags]

The above embodiment has been described that the RF tag 302 is attachedto the worker 300 as shown in FIG. 1 to allow the sensors 2 to specifythe position of the worker 300. However, other methods may be used aslong as the monitoring system 1 can specify the position of the worker300. FIG. 16 is a diagram illustrating another method for specifying theposition of the worker 300. In the example of FIG. 16, an LED 304 thatemits light is attached to a part of each worker. FIG. 16 shows anexample in which an LED 304 is attached to a helmet worn by each worker.

The LED 304 is detected by the cameras 4 rather than by the sensors 2.Thus, the processing of the sensors 2 may be assigned to the cameras 4.According to this configuration, there is no need to provide the sensors2 and therefore it is possible to reduce the number of parts. A lightemission mode of the LED attached to the worker 300 may be set differentfrom a light emission mode of the LED attached to the shiftable robot.This allows the monitoring system to clearly specify the worker 300 andthe shiftable robot without confusion. As long as the position of theworker 300 can be specified, another method may be used without beinglimited to the above methods.

[LEDs Attached to Arm]

The above embodiment has been described that the LEDs are attached tothe arms 406 of the robot 400 as shown in FIG. 8. However, other methodsmay be used as long as the monitoring system 1 can specify the positionsof the arms 406. This modification will be described with reference toan example in which reflective markers are attached to the arms 406. Thereflective markers are markers that reflect when illuminated.

FIG. 17 is a diagram illustrating another method for specifying thepositions of the arms 406. FIG. 17 shows an example in which a firstreflective marker 481, a second reflective marker 482, and a thirdreflective marker 483 are attached to a first arm 411, a second arm 412,a third arm 413, respectively.

An external lighting 700 emits light to obtain reflected light from thereflective markers. The first reflective marker 481, the secondreflective marker 482, and the third reflective marker 483 reflect lightfrom the external lighting 700 in different patterns. For example, thefirst reflective marker 481 reflects a blue color, the second reflectivemarker 482 reflects a yellow color, and the third reflective marker 483reflects a red color. These reflection modes (reflection colors) arestored in advance in the ROM 102.

The second specifying part 8 can recognize the movements of the firstarm 411, the second arm 412, and the third arm 413 on the basis of thereflection modes stored in the ROM 102 in an image inside the combinedarea 600B.

The reflective markers may be configured such that reflected light ofthe reflective markers returns to the camera 4. The reflective markersmay also be configured such that reflected light of the reflectivemarkers scatters.

The above embodiment and this modification have been described that thelight emission or reflection modes of markers attached to the first arm411, the second arm 412, and the third arm 413 are made different inpattern. In the method for making the reflection modes different, theelement in which the reflection modes are made different is not limitedto pattern and the reflection modes may be made different in otherelements. Another element may be, for example, frequency. Anotherexample element may be color. These configurations also allow the secondspecifying part 8 to specify the respective positions of the first arm411, the second arm 412, and the third arm 413.

Modes (for example, light emission modes) of the markers attached to therobot 400 may be different from a mode (for example, a light emissionmode) of the marker attached to the worker 300. This allows themonitoring system to distinguish between the robot 400 and the worker300, such that it is possible to guarantee safety.

[Others]

Robot monitoring systems sometimes use safety fences, safety lightcurtains, safety laser scanners, safety limit switches, or the like.However, since protection areas are fixed, there is a problem that it isnot possible to flexibly respond to layout changes of lines in workareas or the like. On the other hand, the invention can guarantee safetyof workers without using safety fences, safety light curtains, or thelike, and thus can flexibly respond to layout changes.

In addition, to form a protective area three-dimensionally around eachof a plurality of robots using a sensor, it is necessary tocomplicatedly arrange and use a plurality of sensors in threedimensions. On the other hand, the invention can guarantee safety ofworkers without providing sensors respectively for the plurality ofrobots, and thus can simplify the arrangement of sensors.

The processes of the monitoring device 100 are realized by each hardwarecomponent and software executed by the CPU 101. Such software may bestored in advance in the flash memory 104. The software may sometimes bestored in a memory card or other recording medium and distributed as aprogram product. The software may also be provided as a downloadableprogram product by an information provider connected to the Internet.The software is read from the recording medium by an IC cardreader/writer or other reading device or is downloaded via thecommunication IF and then temporarily stored in the flash memory 104.The software is read out from the flash memory 104 by the CPU 101 and isfurther stored in an executable program format in the flash memory 104.The CPU 101 executes the program.

The components of the monitoring device 100 shown in FIG. 2 are generalones. Thus, it can be said that, in one or some of exemplary embodimentsof the invention, the essential part is software stored in the flashmemory 104, a memory card or other recording medium, or software thatcan be downloaded via a network.

The recording medium is not limited to a DVD-ROM, a CD-ROM, a flexibledisk (FD), and a hard disk, and may include a medium that stores aprogram in a static manner such as a magnetic tape, a cassette tape, anoptical disk (for example, a magnetic optical disc (MO), a mini disc(MD), or a digital versatile disc (DVD)), an optical card, asemiconductor memory such as a mask ROM, an electronically programmableread-only memory (EPROM), an electronically erasable programmableread-only memory (EEPROM), or a flash ROM. The recording medium is anon-transitory medium from which the computer can read the program orthe like.

The program referred to herein includes not only a program that isdirectly executable by the CPU but also a program in a source programformat, a compressed program, an encrypted program, or the like.

The embodiments disclosed herein should be considered to be illustrativein all respects and not restrictive. The scope of the invention isindicated by the appended claims rather than by the foregoingdescription, and the invention is intended to include all modificationswithin the concept and scope equivalent to the claims.

What is claimed is:
 1. A monitoring system configured to monitormovement of a worker and a robot to allow the worker to work safely in awork area in which the worker and the robot work, the monitoring systemcomprising: a sensor configured to detect a position of the worker thathas intruded into the work area; a first controller configured tospecify a worker movable area indicating a range in which the worker ismovable on the basis of the position of the worker detected by thesensor; a camera configured to capture an image of an area including atleast the worker movable area specified by the first controller and apredetermined robot occupied area of the robot; and a second controllerconfigured to specify a robot movable area indicating a range in which amovable portion of the robot is movable according to the image of thearea captured by the camera, and further configured to specify a humanbody area of the worker according to the image of the area captured bythe camera, wherein the second controller is further configured to setan area obtained by excluding an area in which the worker cannot bepresent from an area that is a combination of the worker movable areaand the robot occupied area that at least partially overlaps the workermovable area as a combined area, wherein the camera is configured tocapture the image of only the combined area, wherein the firstcontroller is further configured to measure a distance between the robotmovable area specified by the second controller and the human body areaspecified by the second controller, wherein the first controller isfurther configured to restrict movement of the robot when the distancemeasured by the first controller is equal to or less than apredetermined distance.
 2. The monitoring system according to claim 1,wherein the second controller is configured to specify the robot movablearea according to an image of the combined area in the area captured bythe camera, and the second controller-is configured to specify the humanbody area according to the image of the combined area in the areacaptured by the camera.
 3. The monitoring system according to claim 1,wherein the second controller is configured to specify the robot movablearea on the basis of a marker provided on the movable portion of therobot.
 4. The monitoring system according to claim 1, wherein the secondcontroller is further configured to specify a position of the movableportion of the robot according to an image captured by the camera on thebasis of a marker provided on the movable portion of the robot, andwherein the first controller is further configured to compare theposition of the movable portion of the robot specified by the secondcontroller with a movement indicated by movement information input to acontroller of the robot.
 5. The monitoring system according to claim 1,wherein the robot is shiftable, the sensor is further configured tospecify a position of the robot, and the first controller is furtherconfigured to specify the robot occupied area on the basis of theposition of the robot detected by the sensor.
 6. A monitor configured tomonitor movement of a worker and a robot to allow the worker to worksafely in a work area in which the worker and the robot work, themonitor comprising: a first controller configured to specify a workermovable area indicating a range in which the worker is movable on thebasis of a position of the worker detected by a sensor configured todetect the position of the worker that has intruded into the work area;and a second controller configured to specify a robot movable areaindicating a range in which a movable portion of the robot is movableaccording to an image of an area captured by a camera configured tocapture the image of the area including at least the worker movable areaspecified by the first controller and a predetermined robot occupiedarea of the robot, wherein the second controller is further configuredto specify a human body area of the worker according to the image of thearea captured by the camera, wherein the second controller is furtherconfigured to set an area obtained by excluding an area in which theworker cannot be present from an area that is a combination of theworker movable area and the robot occupied area that at least partiallyoverlaps the worker movable area as a combined area, wherein the camerais configured to capture the image of only the combined area, whereinthe first controller is further configured to measure a distance betweenthe robot movable area specified by the second controller and the humanbody area specified by the second controller; and the first controlleris further configured to restrict movement of the robot when thedistance measured by the first controller is equal to or less than apredetermined value.
 7. A monitoring method for monitoring movement of aworker and a robot to allow the worker to work safely in a work area inwhich the worker and the robot work, the monitoring method comprising:specifying a worker movable area indicating a range in which the workeris movable on the basis of a position of the worker detected by a sensorconfigured to detect the position of the worker that has intruded intothe work area; setting an area obtained by excluding an area in whichthe worker cannot be present from an area that is a combination of theworker movable area and a predetermined robot occupied area of the robotthat at least partially overlaps the worker movable area as a combinedarea; capturing an image of only the combined area; specifying a robotmovable area indicating a range in which a movable portion of the robotis movable according to the captured image of only the combined area;specifying a human body area of the worker according to the image of thearea captured by the camera; measuring a distance between the robotmovable area and the human body area; and restricting movement of therobot when the measured distance is equal to or less than apredetermined value.